The present invention pertains to an optical system including a page-oriented holographic means for providing a highly parallel computational device which simulates neural operation. More particularly, the present invention pertains to a system including holographic means comprising a plurality of essentially two-dimensional, spatially localized holograms arranged in an array for projecting interconnection weight encoded beams to interconnect a spatial light modulator which modulates the beams in response to input signals and a detecting means which performs a nonlinear transformation on the modulated beams to produce output signals.
This invention relates to an optical system providing what has become known in the art as a "neural network". Such a network comprises devices that simulate the responses of biological neurons. A model for a neuron N is shown in FIG. 1 to receive three inputs X.sub.1, X.sub.2, and X.sub.3 at a device which sums the inputs according to the simple equation S=X.sub.1 +X.sub.2 +X.sub.3. Positive X's may be defined as excitatory and tend to make the model neuron "fire", that is provide a nonzero output. Negative X's, defined as inhibitory, tend to prevent the model neuron from firing. A nonlinear operator changes an output signal from the output S of the summing device into a new signal according to a particular response curve to be discussed in detail infra. A low input signal to the nonlinear operator, that is a signal below some threshold, S.sub.0, results in a zero output at the nonlinear operator. A high input signal gives a fixed maximum output. An intermediate input results in an intermediate output. Output S' from the nonlinear operator is applied to still other neurons after multiplication by a weighting factor W by a distributor. The signals W.sub.1 S', W.sub.2 S', and W.sub.3 S' are proportional to S' and may be either strong or weak excitatory signals or strong or weak inhibitory signals.
"Technological" or "artificial" neural networks also mimic biological neural networks by arrangement of the neurons in layers as indicated in FIG. 1. The information, memory, and problem solving methods characteristic of the system are determined by the interconnections in the system, that is what is interconnected to what and with what strength. In providing a model neural network, the model neurons are made to communicate laterally on the same layer and to communicate with neurons in other layers.
For various applications, the power of systems simulating neural activity over conventional, sequential machines has been well recognized by the prior art. For instance, Hopfield U.S. Pat. No. 4,660,166 states that many practical problems take such an enormous amount of computation that a solution in real time is not feasible. The Hopfield patent goes on to describe a network which electronically simulates neural activity to provide a system capable of retrieving particular information from a memory in the system, in response to an interrogation of the system. The patentee describes such a retrieval system as an associative memory, that is a memory that provides an output which is in some way associated with a particular input applied to the system. Such an netWork comprises amplifiers characterized by nonlinear, continuous and sigmoidal response curves. The input is processed in parallel. Such networks thus electronically process plural input signals to obtain collective decisional responses to which all of the input signals make a contribution in the range from 0 to 100%.
U.S. Pat. No. 4,752,906 likewise relates to a system employing neural computation to develop sequences of output vectors. This patent describes a neural network as having a highly parallel computational circuit comprising a plurality of electronic amplifiers. Each of the amplifiers feeds back its output signal to itself and to all of the other amplifiers.
Electronic implementations are inherently limited in the number of interconnections that can be made. It appears unlikely that an electronic circuit providing for more than about 1,000,000 i.e. 1.times.10.sup.6 interconnections is feasible. To attempt to attain such a large number of interconnections in an electronic system results in very significant cross-talk problems. Further, electronic systems are seriously limited by power requirements. Such systems are also limited by volume and weight requirements.
The development of optical systems to carry out computations has progressively advanced. In an article by H. J. Caufield, J. A. Neff and W. T. Rhodes, "Optical Computing: The Coming Revolution in Optical Signal Processing", Laser Focus/Electro-Optics, November, 1983, p. 100, earlier progress in the application of optics to mathematical operations is reviewed. Examples of optical apparatus for performing digital matrix multiplication are disclosed in U.S. Pat. Nos. 4,567,569 and 4,809,204.
Further, optical machines that demonstrate different approaches to associative memory have been developed. One approach, developed by the California Institute of Technology, is referred to as photorefractive hologram neural networks. According to this approach, the selectivity of thick holograms in photorefractive materials is used as the primary driver for an optical associative memory. Systems based on this approach have demonstrated significant ability to learn new data. A second approach is represented by U.S. Pat. Nos. 4,739,496 and 4,750,153. In the optical systems disclosed in these patents, multiple, high-resolution images are stored in a holographic medium. When interrogated by an input image, the systems of the two latter patents recall the closest, most correct image stored. Even if these systems are addressed with an incomplete version of one of the stored images, they will output the complete image.
Limitations in the number of interconnections that can be made already have been recognized in optical systems designed according to these first two approaches. The photorefractive holograms used in systems according to the first and second approaches are known in the art as volume holograms which have relatively large thicknesses. Due to the thickness of the holograms, intermodulation noise becomes an increasing factor as the number of interconnections approaches 10.sup.10. Up to now, volume hologram approaches therefore have been limited to less than 10.sup.10 interconnections. When this number of interconnections is approached or exceeded in a system relying upon volume holography, the performance of such system lessens due to increasing problems in the way of lessening dynamic range, increasing intermodulation noise and degeneracy in the interconnections due to multiple order production by each modulation frequency recorded.